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Executive Summary 


Educators have long been concerned about student 
performance in mathematics, particularly for students 
in challenging urban contexts. For this reason, there is 
keen interest in developing and testing interventions 
that improve both instructional capacity and student 
understanding in mathematics. 


In this report, we describe the results of a rigorous 
two-year study of the impacts of a mathematics 
initiative called the Ongoing Assessment Project 
(OGAP) on teacher and student learning in grades 

3-5 in two Philadelphia area school districts. OGAP 

is a mathematics program which combines teacher 
formative assessment practices with knowledge of 
student developmental progressions to build deeper 
student understanding of mathematics content. 
OGAP includes teacher professional development, 
classroom resources, school-based routines for regular 
practice, and ongoing school-based supports. The 
study was conducted in 61 schools during the 2014- 

15 and 2015-16 school years, with OGAP randomly 
assigned to 31 schools and the remaining 30 serving as 
comparison sites. The results of this study showed that 
OGAP produced statistically significant and meaningful 
impacts on both teacher knowledge and student 
learning. 


For the study we collected data on teacher knowledge 
and student learning at three points in time in all 61 
participating schools. The first data collection point, 

or baseline, occurred in the summer of 2014, before 
teachers and their students began involvement in 
OGAP. We also collected data in the spring of 2015, 
after one year of OGAP, and again in the Spring of 
2016, after two years of OGAP. 


Our measure of teacher knowledge is called the TASK, 
or Teacher Assessment of Student Knowledge. The 
TASK is an assessment of teachers’ ability to examine 


student work on an open-ended mathematics problem, 


to analyze the student thinking underlying the work in 


relation to a research-based developmental learning 
trajectory, and to suggest an informed instructional 
response. The measure is distinct from, but closely 
aligned with, the OGAP project. 


Our measures of student impact are two-fold. First, 

we developed a measure of student performance 
called the Learning Trajectory Assessment (LTA). The 
LTA assesses student performance in multiplicative 
reasoning both in terms of the accuracy (correctness) 
of their response and the sophistication of their solution 
process. This assessment is based on the idea that 
increasing student solution sophistication will help 
prepare students for more advanced mathematical 
concepts. The second measure of student impact was 
performance on the annual state test, the Pennsylvania 
System of School Assessment (PSSA) in mathematics. 


The results were based on a rigorous randomized 
control trial (RCT) in which schools were recruited to 
participate in the study and randomlly assigned to 
receive OGAP or serve as a comparison group. OGAP 
schools received two years of OGAP professional 
development, tools, and resources, while the 
comparison schools received $1,000 per year for their 
school activities fund. 


Overall, we found consistent impacts of the Ongoing 
Assessment Project on teachers and students across 
the two years of the study and the different measures. 
The impacts on teachers showed increased knowledge 
on the TASK for teachers who participated in OGAP 
relative to teachers in the comparison group. On the 
LTA aligned assessment, we found significant impacts 
of OGAP on student performance in both accuracy 
and sophistication in both years of the study. We also 
found impacts on PSSA performance in years one and 
cumulatively, but not in year two. The results indicate 
that OGAP can help improve teacher knowledge and 
student learning in urban school districts. 
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Overview 


This report examines the impacts of the Ongoing 
Assessment Project (OGAP) on teacher mathematics 
knowledge and student learning in mathematics in the 
School District of Philadelphia and the Upper Darby 
school district. From 2014-2016, OGAP provided training, 
tools, and resources to participating teachers and 
schools in grades 3-5. Both the program and associated 
research was supported by a grant from the National 
Science Foundation. Year one (2014-2015) began with 
a focus on multiplicative reasoning, which included 
summer professional development, follow-up training, 
and ongoing tools and resources for both teachers 

and teacher leaders. In year two, the emphasis shifted 
to fractions, with similar additional training during the 
school year. In the second year of the intervention, the 
program added three OGAP coaches, who regularly 
supported implementation in approximately 10 schools. 
During both years, teachers also received biweekly 
email reminders about content, formative assessment 
processes, online resources, and teaching suggestions. 


About The Ongoing Assessment 
Project (OGAP) 


The Ongoing Assessment Project (OGAP) has been 
developed and refined over the past 20 years by 
mathematics educators from Vermont. OGAP is 
designed to provide targeted instructional responses 
to improve student learning by combining formative 
assessment practices — an approach that frequently 
assesses student understanding relative to learning 
goals — with contemporary research on how students 
deepen their undrestanding of important mathematics 
concepts. The OGAP process facilitates teachers’ use of 
an ongoing cycle of assessing student understanding, 
analyzing student thinking, and making informed 
instructional responses. The assess-analyze-respond 
cycle is intended to reflect the ongoing nature of the 
teaching and learning process. 


OGAP training, tools, and resources include: 


gm Professional development, most often through 
a summer institute and ongoing school-based 
followup visits throughout the school year. Training 
is focused on developing knowledge of specific 
mathematics topics and the related research base 
on student thinking, as well as training in the use of 
OGAP materials and strategies. 


m OGAP Frameworks which synthesize the problem 
contexts, problem structures, and learning 
trajectories for specific mathematics topics, 
including a visual representation of the learning 
trajectory that can be used to analyze evidence 
in student work and make informed instructional 
decisions. 


mg Electronic item banks and pre-assessments 
comprised of formative assessment tasks 
that are carefully designed to elicit students’ 
developing understandings, common errors, and 
preconceptions or misconceptions. 

m Suggested routines and associated protocols 
for teachers to regularly examine student work 
together in grade-level meetings, or professional 
learning communities (PLCs), and discuss 
instructional strategies. 

g Additional training is provided for a math teacher 
leader, who is expected to support the use of OGAP 
at the school. 


Experimental Study of OGAP 


In the spring of 2014, the research team recruited 
schools in the School District of Philadelphia (SDP) 

and the neighboring district of Upper Darby (UD) to 
participate in a randomized experiment of OGAP. 
Schools were recruited with the promise that they would 
either receive OGAP training, tools, and resources for 
two years (2014-2016) or their school would receive 
$1,000 for their school activity fund each year. In alll, 

61 schools agreed to participate, including 38 schools 
from the SDP, 13 Philadelphia charter schools, and 10 
UD schools. Schools were stratified in each of these 
categories (SDP, charter, and UD schools) and randomly 
assigned to either the OGAP treatment or control 
group. 


Research Design and Analysis Approach 


The research was designed as a randomized control 
trial, which provides the most accurate estimates of 
the causal effects of OGAP on teacher knowledge 
and student learning outcomes. Over the course of 
the two years of OGAP implementation, we assessed 
teachers and students three times. The first, or baseline, 
assessment, occurred in the summer/fall of 2014 

and measured teacher and student knowledge and 
skills before OGAP began. The second assessment 
occurred at the end of year one (spring 2015) and the 
third assessment occurred in the spring of 2016. These 
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allowed us to conduct three analyses of the impacts of 
OGAP on teachers and students. These include a year 
one impact (2014 baseline to spring 2015), a year two 
impact (spring 2015 to spring 2016) and, for students, 
the two-year impacts of OGAP on those students who 
remained in their schools over the two years of the 
study. 


Our analytic approach focused on estimating the 
effects of OGAP on teachers and students in the 
schools receiving OGAP training, tools, and resources 
in contrast to the teachers and students in the schools 
in the comparison group. Our models estimate the 
effects of OGAP on teachers and students, controlling 
for prior performance and demographic and school 
characteristics. The models appropriately nest teachers 
and students within schools to more accurately 
account for the hierarchical nature of educational 
contexts. 


Sample 


The study included about 14,000 students in almost 

700 classrooms in 61 schools (Table 1). The 61 schools 
included 38 from the School District of Philadelphia 
(SDP), 10 from Upper Darby (a school district west 

of the city of Philadelphia), and 13 charter schools 
located in Philadelphia. Overall, the demographics of 
the teachers, students, and schools in the OGAP and 
comparison sites were quite similar, although there 
were a few important differences, which are described 
below and detailed in Appendix A. 


Table 1 
Sample Sizes of Teachers, Students, and Schools 


Participants OGAP Comparison Total 
Grades 3-5 Teachers 347 331 678 
Grades 3-5 Students 6,737 7,251 13,988 


Schools 31 30 61 


The two groups of grades 3-5 teachers had similar 
experience (about 13 years), and similar proportions 
of ELL (about 11%) and teachers who taught special 
education students (about 20%). The OGAP teachers 
had slightly higher average teacher knowledge scores 
at baseline (2.16 compared to 2.09). 


Of the grades 3-5 students, there were more Black 
and Asian students in the OGAP schools, but fewer 
Hispanic students. The percentage of students on free/ 


reduced lunch was similar for both groups (about 69%). 
The two groups had similar percentages of English 
language learners and special education students. 
The two groups were also similar on their baseline test 
performance, although the OGAP students had slightly 
higher solution sophistication scores (1.21 compared to 
1.17} 


Finally, the OGAP and comparison schools were similar 
in size (around 600 students) and had similarly high 
proportions of students on free/reduced lunch (more 
than 80%). 


Impact of OGAP on Teachers 


Teacher impacts were assessed on an assessment 
called the Teacher Analysis of Student Knowledge 
(TASK). The TASK is a grade-specific, online assessment 
for mathematics teachers which measures important 
components of the instructional knowledge necessary 
to teach to the high expectations of the Common Core 
State Standards in Mathematics. The TASK provides 
teachers with a grade-appropriate, open-ended 
mathematics problem and a set of student responses, 
and asks teachers to (1) analyze the thinking of the 
students based on their responses; (2) rank each 
student's solution based on the level of sophistication of 
the mathematical thinking represented, and explain the 
rationale for the rankings given to each student, and; 
(3) suggest instructional next steps, and their rationale, 
for a subset of the students. The TASK was developed 

by CPRE in 2012 and has been used in a variety of 

state and district contexts. As a measure of learning 
trajectory-oriented formative assessment, it is highly 
aligned with OGAP. 


Figure 1 shows the impact of OGAP professional 
development, tools, and resources on Teacher 
Knowledge of Student Thinking in multiplication for 

a typical SDP teacher. The graph illustrates that SDP 
teachers who participated in OGAP performed about a 
half point (or 13%) better on the four point multiplication 
TASK than did teachers who did not participate in 
OGAP. The values represented in the graph are those 
for a regular education (i.e. not soecial education) 
teacher of average experience from a school of 
average size with average student demographics and 
average percentage of students on free/reduced 
lunch. 


In the first year of OGAP, which focused solely on 
multiplication, teachers in schools implementing OGAP 
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Figure 1 
OGAP Impacts on Teacher Knowledge of Student 
Thinking (TASK) 
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significantly outperformed teachers in the comparison 
schools. In year two, OGAP teachers continued to 
outgain comparison teachers in TASK performance in 
multiplication, even as the program shifted its emphasis 
toward fractions. The third set of bars in the graph 
shows similar effects for teachers who remained in their 
schools over the two years of the study, and thus had 
data from 2014 to 2016. The complete models from 
which the graph was derived is shown in Appendix B. 


Impact of OGAP on Students 


We assessed the impacts of OGAP students on two 
different assessments. The first assessment was the 
Learning Trajectory Assessment (LTA). The LTA assesses 
student performance in multiplicative reasoning on two 
dimensions — accuracy and sophistication. Accuracy 
refers to the correctness of a student response, while 
sophistication refers to the congnitive complexity 

of the solution approach that students use. Student 
solution sophistication is an important dimension to 
measure because it reflects students’ ability to master 
increasingly challenging mathematical content. For 
example, consider two students who answer a single 
digit multiplication problem correctly. One student uses 


an inefficient additive approach to solving the problem, 


while the other uses a multiplicative approach. Just 
by looking at their correct answers, we would not be 
able to distinguish between the two students. But in 
terms of sophistication, the second student will be 
more prepared for more advanced problems, like 
double digit multiplication, than the first student. This 
is both an example of the importance of measuring 
solution sophistication in addition to accuracy, as well 


as an example of the core approach of OGAP, which 
is to use this knowledge to help students progress 
developmentally. 


The second assessment we analyzed was student 


performance on the Pennsylvania System of School 
Assessment (PSSA). Although only a portion of the PSSA 
is focused on multiplication and fractions in grades 
3-5, and thus not a perfectly aligned measure of 
OGAP impacts, it is the state test in Pennsylvania, and 
therefore more meaningful to district leaders than the 
LTA. 


The Learning Trajectory 
Assessment 
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Student Impacts on the Learning 
Trajectory Assessment 


Overall, OGAP produced significant improvements 

in both student accuracy and sophistication on the 
Learning Trajectory Assessment. As shown in Figures 

2 and 3, students in schools participating in OGAP 
performed significantly better than did students in the 
control schools. 
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Figure 2 
OGAP Impacts on on 4th Grade Student Accuracy/ 
Correctness Learning Trajectory Assessment 
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Overall, the students in the OGAP schools had 
significantly higher percentages of correct/accurate 
responses on the LTA in both the first and second years 
of participation in OGAP, as well as for the subgroup of 
students who were in their schools over the two years 
of OGAP. To illustrate these differences, Figure 2 shows 
the average differences in 4th grade performance 

of typical students from a school with average 
demographics in the School District of Philadelphia. 
Students in the OGAP schools had a significantly 
greater average correctness score on the LTA in both 
2014-15 and 2015-16, although the differences were 
bigger in year one than year two. The average effect 
for the typical student who was in 3rd grade in an 
OGAP school in 2014-15 and stayed in their OGAP 
school in 4th grade in 2015-16 was about a 10 percent 
increase in performance. It is also important to temper 


these differences by pointing out that, even though 
OGAP was improving student performance on the LTA, 
students were still only getting less than half of the items 
correct. 


Students in the OGAP schools also outperformed 
students in the comparison schools on the sophistication 
dimension of the LTA assessment. Generally, students 

in the schools participating in OGAP had solution 
approaches that were about 10 percent more 
sophisticated, on average. An example of these 

results for average 4th grade SDP students from typical 
schools is shown in Figure 3. In 2014-15 and 2015-16, 

the OGAP students had solution approaches that 

were significantly more sophisticated than did the 
comparable 4th grade students in the comparison 
schools. Although OGAP students were using 
significantly more sophisticated solution strategies, it is 
important to recognize there is still substantial room for 
growth, as these responses still represent mostly additive 
or early multiplicative solution approaches. 


More detailed results of students of different 
backgrounds and in different school conditions are 
shown in multi-level regression results in Appendix C. A 
few additional important details emerged from these 
results that are worth mentioning. First, girls performed 
significantly lower than boys on both the accuracy 
and solution sophistication measures. Second, students 
in charter schools performed worse in year 1 than did 
students in the SDP schools, but performed no different 
than students in other SDP schools in the 2nd year or 
across the two-year stable sample. 


Student Impacts on the PSSA 


The final analysis examined the impacts of OGAP on 
the Pennsylvania System of School Assessment (PSSA). 
Depending on the grade level, the PSSA contains 
different amounts of emphasis on multiplication, which 
was the main focus of OGAP during the 2014-2016 work 
in Philadelphia area schools (as well as an introduction 
of fractions in the second year of the intervention). 
Therefore, the PSSA covers content beyond the focus of 
OGAP and is therefore a less aligned assessment. 


Even given these caveats, there are significant effects 
of OGAP on student PSSA performance. 


Figure 4 shows the estimated impacts of OGAP on 
typical SDP 4th graders in average school contexts. As 
shown, 4th graders in OGAP schools scored an average 
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Figure 4 
OGAP Impacts on on 4th Grade PSSA Scores 
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of 966 on the PSSA in 2015, compared to 955 for 
students in the comparison schools, a difference of 11 
points. There was a non-significant difference of seven 
points in 2015-2016 between the typical fourth grader in 
an OGAP SDP school versus a student in a comparison 
school. The stable sample of 3rd grade students who 
were in an OGAP school in 2014-15 and stayed in 

that school as 4th graders in the 2015-16 school year 
were predicted to have a 17 point boost in PSSA 
performance relative to students in the comparison 
schools. For those interested in the statistical model 
underpinning these results, the details of these impacts 
are shown in Appendix D. 


Finally, these results can be examined in terms of 
percentile gains. From an examination of the content 
areas covered on the PSSA by grade (documentation 
publicly available on the Pennsylvania Department 

of Education’s webpage) and in consultation with 
math content experts, we believe that 25-50 percent 
of the PSSA (depending on grade) does not cover 
multiplication and division. We therefore estimate that, 
on the proportion of the PSSA focused on these two 
mathematical topics, OGAP produced about 6-11 
percentile point gains for students, as shown in Table 2. 


Table 2 

Average percentile gains for student particpating in 
OGAP for two years 

Percentile Point 


A student inthis Would be predicted 


percentile to move to this Gain 
percentile 
10th 19th 
25th 40th 15 
Soth 58th 
75th 80th 


Conclusion 


The Ongoing Assessment Project (OGAP) is a set 

of professional development strategies, tools, 

and resources which help teachers use principles 
of formative assessment and research-based 
developments in student mathematical learning to 
develop students mathematical understanding in a 
range of math domains. 


During the two school years from 2014 to 2016, OGAP 
was implemented in 61 public and charter schools in 
the School District of Philadelphia, as well as the nearby 
school district of Upper Darby. In partnership with OGAP 
program staff and the participating school districts, the 
Consortium for Policy Research in Education conducted 
a randomized control trial (RCT) to experimentally 
examine the impacts of OGAP on teachers’ knowledge 
and students’ learning. 


The results show statistically significant impacts of OGAP 
on both teacher knowledge and student performance. 
Teachers in the OGAP schools demonstrated 
significantly higher capability to analyze student 
thinking as represented in student work samples and 
make informed instructional responses, as measured on 
a validated authentic assessment called the Teacher 
Assessment of Student Knowledge (TASK). 


The students of OGAP teachers also signfiicantly 
outperformed students in a set of comparison schools 
on two measures of their performance. The first 
measure, called the Learning Trajectory Assessment, 
assessed students’ correctness as well as solution 
sophistication in multiplicative reasoning, an important 
aspect of student conceptual understanding. 

The second measure was the state test used in 
Pennsylvania, the Pennsylvania System of School 
Assessment. Across both years of the study and for the 
sub-sample of students who remained in their schools 
over the two years of the study, the OGAP students 
performed higher on both measures than did students 
in the comparison schools. 


In sum, the findings from this rigorous study indicate 
that OGAP produces statistically significant and 
educationally meaningful impacts on both teacher 
knowledge and student learning. 
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Appendix A 


Descriptive Student and School Characteristics, 2014-2015 


Student Characteristics irae Pe 
Fall 2014 Correctness 49 48 
(.52) (.50) 
Fall 2014 Sophistication 1.17 1.21 + 
(1.24) (1.24) 
% Female 48.66 48.92 
J%o White 17.40 18.43 
% Black 49.20 51.04 * 
% Hispanic 19.737" 11.81 
% Asian 7.05 13.80 *** 
% Other 6.12 ** 4.9) 
% Free/Reduced Lunch 69.14 68.05 
% Special Education 18.35 17.36 
% English language learners 10.95 11.97 + 
Comparison OGAP 
Teacher Characteristics (N=331) (n=347) 
2014 TASK Score 2.09 2.16 + 
(.025) (.025) 
Years of Experience 12.75 12.311 
(.47) (.44) 
% English language learner Teachers 11.42 12.07 
% Special Education Teachers 17.69 23.87 + 
Comparison OGAP 
School Characteristics (n=30) (n=31) 
School Size (hundreds) 5.94 5.67 
(1.88) (2.25) 
% Free/Reduced Lunch 81.42 84.12 
(21.28) (15.89) 
Charter Schools 6 7 
Upper Darby Schools 5 5 
Philadelphia Schools 19 19 


+ p<.10, * p< .05, ** p<.01, *** p<.001 


1 Sample size varies by variable due to missing data 
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Appendix B 


Impact of OGAP on Teacher Knowledge of Student Thinking (TASK) in Multiplication 


Year 1 Year 2 Two Year Stable Sample 
Variable 2014-15 2015-16 2014-16 
(n=435) (n=291) (n=246) 
Constant 1.077** Liar 1.284*** 
(.195) (.258) (.322) 
Pre-Test 7 2*** A53*** A24*** 
(.062) (.049) (.065) 
OGAP oor" AoE 318%" 
(.046) (.066) (.083) 
Multiplication s183*** :194*** 
(.050) (.054) 
Years of Experience .0001 -.007+ -.005 
(.004) (.004) (.005) 
ELL Teacher -.004 .029 .105 
(.062) (.111) (.155) 
Special Ed. Teacher 021 -.192** -.100 
(.063) (.064) (.120) 
School-Level Variables 
School Size (hundreds) -.001 -.010 -.008 
(.008) (.016) (.017) 
% Free/Reduced Lunch .086 .263 .046 
(.173) (.403) (.425) 
Charter School -.0] .102 -.063 
(.090) (.156) (.168) 
Upper Darby School .026 .081 -.074 
(.100) (.136) (.172) 
Percent Black Students .034 -.307 -.16] 
(.114) (.294) (.291) 
Percent Hispanic Students .059 -.292 .002 
(.176) (.324) (.349) 
Percent Asian Students .632** -.166 462 
(.192) (.439) (.499) 
Percent Multirace Students -.896 Vd -1.384 
(.933) (1.338) (1.663) 


* p<.10, * p<.05, ** p<.01, *** p<.001 
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Appendix C 
Impact of OGAP on LTA, Accuracy and Sophistication 


Accurancy/Correctness Sophistication 
Two Year Two Year 
Year 1 Year 2 Stable Sample Year 1 Year 2 Stable Sample 
2014-15 2015-16 2014-16 2014-15 2015-16 2014-16 
Variable (n=9,099) (nN=7,162) (n=4,508) (nN=9,099) (n=7,162) (n=4,508) 
Constant 5075" .368*** A89*** oAa* 208" 3.000" * 
(.046) (.035) (.044) (.191) (.206) (.267) 
Pre-Test 22 A46*** 2018" A62*** ROY a aiid Ay 2et* 
(.01) (.019) (.013) (.017) (.027) (.025) 
OGAP .049** .024+ .043** 245*** .131* 259** 
(.016) (.013) (.016) (.068) (.065) (.099) 
Third Grade 035** 02 =3297** -.248*** 
(.011) (.015) (.061) (.073) 
Fifth Grade (Grades 4-5 for stable -.058*** 011 011 -.029 .003 121+ 
sample) (.012) (.011) (.013) (.068) (.072) (.063) 
Female -.01* -.O11+ =025*** -.098*** -.048* -.109*** 
(.005) (.006) (.007) (.020) (.023) (.031) 
Black -.084*** =057*** -.08*** =350*"* 225 4e** -. 408*** 
(.012) (.016) (.022) (.054) (.070) (.108) 
Hispanic -.044** -.024 -.038 -.180** -.124+ -.151 
(.014) (.017) (.028) (.067) (.072) (.134) 
Asian .089*** (093"** 2)" ot AS?*** :619"** 
(.016) (.023) (.03) (.081) (.084) (.138) 
Other Ethnicity -.021 -.03+ -.045* -.129+ -.125* -.292** 
(.016) (.015) (.022) (.072) (.069) (.104) 
Free/Reduced Lunch -.018** -.018** -.021* -,]22*** -.048+ -.108* 
(.006) (.006) (.01) (.030) (.031) (.046) 
Special Ed. Students =1287** -.07*** =i te" -.584*** -. 389 *** -.588*** 
(.009) (.01) (.013) (.036) (.055) (.079) 
English language learners =071*** -.045*** -.045*** -.327*** -.212*** -.31]1*** 
(.009) (.008) (.012) (.045) (.039) (.065) 
School-Level Variables 
School Size (hundreds) -.007 -.003 -.005 -.028 -.027+ -.038 
(.004) (.003) .003) (.017) (.015) (.025) 
% Free/Reduced Lunch =165*** =,1677** -.183*** -.676*** -.767*** -1.102*** 
(.033) (.035) (.034) (.151) (.174) (.203) 
Charter School -.032+ -.003 -.01] -.186* -.062 -.138 
(.018) (.02) (.025) (.074) (.104) (.129) 
Upper Darby School -.035+ 01] .007 -.167* .072 031 
(.019) (.016) (.016) (.074) (.087) (.090) 


+ p<.10, * p<.05, ** p<.01, *** p<.001 
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Appendix D 
Impact of OGAP on PSSA 


Year 1 Year 2 Two Year Stable 
Variable 2014-15 2015-16 Sample 2014-16 
(n=9,099) (Nn=7,162) (n=4,508) 
Conshant 1001.52 *** 962.04 994.5] 
(12.91) (22.89) (27.79) 
Pre-Test F399: 8** 102.34 *** 74.19 *** 
(2.037) (5.51) (3.84) 
OGAP 11.10 * 6.65 17.04 + 
(5.44) (8.31) (9.39) 
Third Grade -10.00 ** 14.44 ** 
(3.18) (5.58) -- 
Fifth Grade (Grades 4-5 for stable sample) -37.80 *** 11.71 ** 13:33: ** 
(3.11) (4.03) (4.82) 
Female -5.69 *** -10.16 *** -7.62 * 
(1.36) (2.33) (3.29) 
Black -30.92 *** -32.68 *** -46.03 *** 
(5.13) (6.31) (11.47) 
Hispanic -16.02 *** -18.19 *** -26.84 * 
(4.80) (5.62) (11.85) 
Asian 36.77 *** 33.64 *** 36.59 ** 
(5.93) (9.03) (14.09) 
Other Ethnicity -9.Al + -12.58 * -13.62 
(5.07) (5.95) (10.41) 
Free/Reduced Lunch -14.91 *** -13.69 *** -6.91 + 
(2.70) (2.88) (3.95) 
Special Education Students -39.66 *** -38.74 *** -48.65 *** 
(2.84) (4.17) (5.63) 
English language learners -38.13 *** “37:25. "** -35.26 *** 
(3.35) (4.77) (6.71) 
School-Level Variables 
School Size (hundreds) -0.75 -1.47 -1.57 
(1.06) (1.49) (1.84) 
% Free/Reduced Lunch -70.21 *** -75.89 *** -82.83 *** 
(12.45) (22.38) (20.41) 
Charter School 8.10 
(13.31) -- -- 
Upper Darby School 8.97 21.537? 
(6.70) (7.62) -- 


+ p<.10, * p<.05, ** p<.01, *** p<.001 


Note: The pre-test used in the PSSA analyses was the 2014 student score for accuracy/correctness. We did this because there 
would otherwise have been no first year and stable sample baseline for 3rd graders, as there is no end of second grade PSSA 


assessment. 
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